ci: constrain internal builds to exact local wheels - #2510
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
/ok to test |
a571b64 to
472b7cd
Compare
472b7cd to
f39b4c0
Compare
|
f39b4c0 to
5fcc5fd
Compare
3e8866f to
9eb4470
Compare
|
@kkraus14 This PR now keeps dependency selection entirely in CI, uses exact constraints for both pathfinder and bindings across every in-scope workflow, and no longer changes either build hook. I plan to merge after clean CI unless you see a remaining blocker. (At this moment the CI is very unreliable b/o infrastructure issues; I'll keep retrying; we had a clean run before I merged 2520, therefore I believe it should work again.) |
kkraus14
left a comment
kkraus14
left a comment
There was a problem hiding this comment.
Reposting comment I made offline:
There is so much unnecessary added complexity here. We shouldn't need a whole python helper for constraint file generation. We already know the exactly wheel we built earlier for dependencies so we can easily just pass that information through in the GitHub Action workflows.
9eb4470 to
2126160
Compare
|
@kkraus14, I removed the Python helper and moved the constraint generation directly into the workflows, following your suggestion. CI is running, and I've updated the PR description to reflect the new implementation. Could you please take another look? |
| - name: Constrain cuda.core to the local cuda.bindings wheel | ||
| run: | | ||
| pathfinder_wheels=(cuda_pathfinder/cuda_pathfinder-*.whl) | ||
| bindings_wheels=("${CUDA_BINDINGS_ARTIFACTS_DIR}"/cuda_bindings-"${BUILD_CUDA_MAJOR}".*.whl) | ||
| test "${#pathfinder_wheels[@]}" -eq 1 | ||
| test "${#bindings_wheels[@]}" -eq 1 | ||
| test -f "${pathfinder_wheels[0]}" | ||
| test -f "${bindings_wheels[0]}" | ||
| mkdir -p wheel-constraints | ||
| if [[ "${{ inputs.host-platform }}" == win* ]]; then | ||
| pathfinder_uri="file:///$(cygpath -am "${pathfinder_wheels[0]}")" | ||
| bindings_uri="file:///$(cygpath -am "${bindings_wheels[0]}")" | ||
| else | ||
| pathfinder_uri="file:///host$(realpath "${pathfinder_wheels[0]}")" | ||
| bindings_uri="file:///host$(realpath "${bindings_wheels[0]}")" | ||
| fi | ||
| { | ||
| printf 'cuda-pathfinder @ %s\n' "${pathfinder_uri}" | ||
| printf 'cuda-bindings @ %s\n' "${bindings_uri}" | ||
| } | tee wheel-constraints/cuda-core.txt | ||
|
|
There was a problem hiding this comment.
I thought we explicitly want to test cuda-core against the already-published cuda-bindings to make sure we don't break backward compat. Does this remove that kind of testing (in favor of testing with latest)? I think ideally we need to do both.
4 participants
Description
Fixes #2468.
xref: #2464, #2470
Same-checkout CI builds must use the exact CUDA Python wheel artifacts selected by the workflow. A compatible package from PyPI must not satisfy an internal build dependency when CI intends to exercise a local artifact.
This PR generates PEP 508 direct-reference constraints for the selected
cuda-pathfinderandcuda-bindingswheels, for example:The workflows apply each constraint file through both
PIP_BUILD_CONSTRAINTandPIP_CONSTRAINT. This constrains dependencies installed into PEP 517 isolated build environments as well as other pip dependency resolution in the build step, while leaving package indexes available for unrelated third-party dependencies.PRs #2509 and #2520 report the versions and import locations of build dependencies. Those diagnostics make the selected artifacts visible in CI logs; the constraints in this PR enforce their selection.
Dependency constraints
The protected build edges are:
cuda.bindingscuda-pathfinderwheelcuda.corecuda-pathfinderwheel and onecuda-bindingswheel matching the current CUDA majorcuda.corecuda-pathfinderwheel and one downloadedcuda-bindingswheel matching the previous CUDA majorEach constraint-generation step verifies that exactly one matching artifact exists before starting the consumer build. The previous-major bindings artifact is kept in a separate directory so current- and previous-major wheels cannot become an ambiguous input set.
The direct references use paths appropriate to each execution environment:
/hostmount.cygpathfor Git Bash and native pip subprocesses.Affected environments install pip 25.3 or newer for build-constraint support.
Workflow coverage
The constraints are applied consistently across:
cuda.corebuild;The policy is contained entirely in CI configuration. Package build requirements and ordinary builds without these constraint variables are unchanged.